Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[fix](nereids)let query be able to forwarded to master #40797

Merged
merged 2 commits into from
Sep 14, 2024

Conversation

starocean999
Copy link
Contributor

Proposed changes

pick from master #40119

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@starocean999
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 49219 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit d32b10b8ab953d7ba022ba1b1b18659719469ab3, data reload: false

------ Round 1 ----------------------------------
q1	17567	4388	4335	4335
q2	2035	186	144	144
q3	10233	1899	1926	1899
q4	10336	1229	1332	1229
q5	8557	3876	3936	3876
q6	235	122	129	122
q7	2020	1602	1605	1602
q8	9505	2733	2708	2708
q9	10501	10059	9985	9985
q10	8602	3504	3490	3490
q11	422	248	251	248
q12	465	305	293	293
q13	18345	3974	4064	3974
q14	355	320	334	320
q15	502	453	454	453
q16	528	453	458	453
q17	1133	970	947	947
q18	7279	6857	6883	6857
q19	1680	1602	1543	1543
q20	544	312	303	303
q21	4363	4195	4057	4057
q22	506	381	392	381
Total cold run time: 115713 ms
Total hot run time: 49219 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4326	4274	4298	4274
q2	318	226	222	222
q3	4209	4114	4106	4106
q4	2757	2739	2745	2739
q5	7173	7146	7115	7115
q6	235	123	120	120
q7	3167	2853	2868	2853
q8	4397	4522	4522	4522
q9	14242	13886	13936	13886
q10	4197	4245	4217	4217
q11	777	675	677	675
q12	1017	856	849	849
q13	6280	3751	3720	3720
q14	460	422	429	422
q15	487	459	454	454
q16	630	608	592	592
q17	3829	3832	3784	3784
q18	8676	8711	8871	8711
q19	1723	1650	1633	1633
q20	2362	2105	2144	2105
q21	8431	8463	8640	8463
q22	1020	921	902	902
Total cold run time: 80713 ms
Total hot run time: 76364 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 212857 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit d32b10b8ab953d7ba022ba1b1b18659719469ab3, data reload: false

query1	961	422	379	379
query2	6531	2278	2080	2080
query3	6918	210	208	208
query4	23251	21866	21657	21657
query5	19729	6559	6494	6494
query6	286	224	241	224
query7	4332	301	305	301
query8	257	299	255	255
query9	3076	2640	2585	2585
query10	472	311	302	302
query11	15569	15320	15176	15176
query12	127	80	72	72
query13	1033	441	431	431
query14	17533	13376	13675	13376
query15	362	215	231	215
query16	6486	285	264	264
query17	1937	924	925	924
query18	890	323	317	317
query19	208	151	151	151
query20	83	80	81	80
query21	190	97	96	96
query22	5237	5113	5043	5043
query23	34403	33458	33592	33458
query24	7830	6320	6355	6320
query25	531	425	440	425
query26	1273	160	167	160
query27	2432	294	291	291
query28	6046	2272	2230	2230
query29	2933	2778	2628	2628
query30	244	169	168	168
query31	961	726	720	720
query32	77	62	64	62
query33	451	254	267	254
query34	859	467	482	467
query35	1165	919	982	919
query36	1271	1075	1379	1075
query37	175	64	59	59
query38	3087	2909	2933	2909
query39	1379	1323	1325	1323
query40	299	95	96	95
query41	39	38	47	38
query42	88	90	84	84
query43	615	625	651	625
query44	1154	736	728	728
query45	249	228	228	228
query46	1229	966	984	966
query47	1877	1797	1771	1771
query48	500	420	415	415
query49	640	369	376	369
query50	847	594	623	594
query51	4714	4687	4684	4684
query52	96	75	73	73
query53	233	192	178	178
query54	2677	2506	2472	2472
query55	88	80	91	80
query56	223	199	217	199
query57	1178	1171	1189	1171
query58	229	216	225	216
query59	3465	3267	3241	3241
query60	241	205	202	202
query61	100	97	99	97
query62	817	466	462	462
query63	197	178	177	177
query64	3553	1662	1456	1456
query65	3598	3552	3582	3552
query66	768	429	428	428
query67	15510	15621	15310	15310
query68	9493	647	648	647
query69	493	275	281	275
query70	1770	1225	1537	1225
query71	400	304	307	304
query72	6843	4804	4802	4802
query73	741	325	326	325
query74	6317	5856	5849	5849
query75	5297	3706	3626	3626
query76	5350	1151	1189	1151
query77	934	259	258	258
query78	12724	12377	11535	11535
query79	10985	629	653	629
query80	777	367	378	367
query81	495	244	240	240
query82	1060	104	95	95
query83	169	138	134	134
query84	259	70	73	70
query85	887	332	326	326
query86	338	291	289	289
query87	3261	3061	3027	3027
query88	5008	2392	2383	2383
query89	447	291	283	283
query90	1972	210	209	209
query91	162	127	134	127
query92	63	54	50	50
query93	5648	596	608	596
query94	737	211	209	209
query95	2053	2114	1941	1941
query96	660	345	334	334
query97	6580	6517	6438	6438
query98	225	201	215	201
query99	3064	903	970	903
Total cold run time: 321021 ms
Total hot run time: 212857 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.82 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit d32b10b8ab953d7ba022ba1b1b18659719469ab3, data reload: false

query1	0.02	0.02	0.02
query2	0.07	0.03	0.02
query3	0.26	0.04	0.04
query4	1.80	0.06	0.08
query5	0.56	0.52	0.52
query6	1.23	0.62	0.61
query7	0.01	0.01	0.01
query8	0.03	0.02	0.03
query9	0.52	0.51	0.49
query10	0.55	0.55	0.53
query11	0.12	0.08	0.08
query12	0.12	0.09	0.10
query13	0.62	0.60	0.62
query14	0.78	0.79	0.79
query15	0.78	0.76	0.75
query16	0.36	0.37	0.37
query17	1.01	0.96	1.01
query18	0.23	0.25	0.24
query19	1.94	1.78	1.87
query20	0.02	0.01	0.01
query21	15.47	0.58	0.55
query22	1.92	2.15	1.89
query23	17.29	0.95	0.80
query24	4.68	1.47	0.96
query25	0.38	0.06	0.05
query26	0.55	0.16	0.15
query27	0.05	0.04	0.03
query28	8.26	0.75	0.73
query29	12.60	2.36	2.33
query30	0.61	0.54	0.53
query31	2.81	0.39	0.38
query32	3.39	0.50	0.49
query33	3.05	3.05	3.09
query34	15.26	4.79	4.82
query35	4.86	4.87	4.86
query36	1.05	1.02	1.00
query37	0.06	0.05	0.04
query38	0.04	0.02	0.02
query39	0.02	0.01	0.01
query40	0.16	0.14	0.15
query41	0.07	0.01	0.01
query42	0.02	0.01	0.01
query43	0.02	0.02	0.01
Total cold run time: 103.65 s
Total hot run time: 30.82 s

@doris-robot
Copy link

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit d32b10b8ab953d7ba022ba1b1b18659719469ab3 with default session variables
Stream load json:         20 seconds loaded 2358488459 Bytes, about 112 MB/s
Stream load orc:          58 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet:      31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select:       22.2 seconds inserted 10000000 Rows, about 450K ops/s

@starocean999
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 49536 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 00c523ca83a9caf824d6a1fafbc7ddff1f31ac5d, data reload: false

------ Round 1 ----------------------------------
q1	17813	4413	4372	4372
q2	2081	155	150	150
q3	10264	1934	1926	1926
q4	10135	1253	1329	1253
q5	8487	3930	3963	3930
q6	231	124	125	124
q7	2036	1595	1601	1595
q8	9339	2757	2724	2724
q9	10303	10129	9780	9780
q10	8646	3556	3534	3534
q11	433	252	255	252
q12	467	292	297	292
q13	18337	4024	4053	4024
q14	350	324	333	324
q15	509	468	459	459
q16	551	459	483	459
q17	1140	992	976	976
q18	7310	7030	6985	6985
q19	1695	1577	1573	1573
q20	516	322	294	294
q21	4428	4179	4118	4118
q22	495	392	404	392
Total cold run time: 115566 ms
Total hot run time: 49536 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4324	4321	4296	4296
q2	319	226	222	222
q3	4190	4160	4143	4143
q4	2757	2762	2740	2740
q5	7160	7131	7109	7109
q6	235	117	119	117
q7	3250	2867	2856	2856
q8	4416	4503	4531	4503
q9	13733	13676	13681	13676
q10	4248	4309	4294	4294
q11	777	711	673	673
q12	1049	864	853	853
q13	6558	3727	3751	3727
q14	456	434	416	416
q15	507	471	459	459
q16	629	567	605	567
q17	3789	3802	3874	3802
q18	8845	8805	8762	8762
q19	1744	1708	1681	1681
q20	2431	2155	2148	2148
q21	8484	8525	8405	8405
q22	1022	970	943	943
Total cold run time: 80923 ms
Total hot run time: 76392 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 212343 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 00c523ca83a9caf824d6a1fafbc7ddff1f31ac5d, data reload: false

query1	918	393	419	393
query2	6560	2081	2060	2060
query3	6921	207	206	206
query4	23419	21565	21484	21484
query5	19750	6571	6536	6536
query6	289	212	221	212
query7	4328	305	318	305
query8	268	253	282	253
query9	3042	2640	2541	2541
query10	462	307	302	302
query11	16287	15115	15151	15115
query12	123	73	72	72
query13	1018	447	432	432
query14	17009	13413	13313	13313
query15	370	212	237	212
query16	6469	296	269	269
query17	1743	929	909	909
query18	889	316	314	314
query19	207	152	149	149
query20	81	73	77	73
query21	192	102	93	93
query22	5225	5147	5031	5031
query23	34225	33551	33522	33522
query24	7727	6327	6329	6327
query25	529	441	409	409
query26	1272	159	162	159
query27	2299	295	294	294
query28	6039	2257	2236	2236
query29	2881	2742	2660	2660
query30	248	172	167	167
query31	963	737	753	737
query32	71	65	58	58
query33	459	266	275	266
query34	865	484	457	457
query35	1135	940	945	940
query36	1319	1150	1291	1150
query37	171	60	61	60
query38	3050	2946	2907	2907
query39	1379	1318	1321	1318
query40	310	95	94	94
query41	40	37	36	36
query42	85	95	79	79
query43	683	560	595	560
query44	1200	717	722	717
query45	246	231	229	229
query46	1224	957	983	957
query47	1944	1936	1856	1856
query48	521	420	416	416
query49	650	368	383	368
query50	865	581	603	581
query51	4754	4668	4631	4631
query52	99	77	82	77
query53	232	198	180	180
query54	2681	2483	2483	2483
query55	82	85	79	79
query56	241	210	218	210
query57	1278	1115	1035	1035
query58	226	205	215	205
query59	3430	3389	3181	3181
query60	229	207	201	201
query61	97	97	97	97
query62	834	497	474	474
query63	203	170	174	170
query64	3545	1631	1476	1476
query65	3622	3570	3531	3531
query66	774	424	423	423
query67	16306	15327	15464	15327
query68	9382	647	653	647
query69	489	267	266	266
query70	1476	1299	1458	1299
query71	400	317	315	315
query72	6789	4868	4826	4826
query73	746	328	323	323
query74	6315	5848	5833	5833
query75	5123	3681	3682	3681
query76	5102	1128	1194	1128
query77	844	263	253	253
query78	12616	11865	11509	11509
query79	8328	638	641	638
query80	1810	391	383	383
query81	500	241	243	241
query82	1441	103	98	98
query83	179	140	135	135
query84	265	72	69	69
query85	894	320	317	317
query86	328	308	286	286
query87	3271	3022	3001	3001
query88	4654	2361	2405	2361
query89	492	311	293	293
query90	1946	213	212	212
query91	166	125	135	125
query92	59	53	49	49
query93	6442	598	579	579
query94	735	212	204	204
query95	2007	1991	1936	1936
query96	658	331	338	331
query97	6493	6336	6428	6336
query98	228	214	201	201
query99	3107	966	854	854
Total cold run time: 319888 ms
Total hot run time: 212343 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 31.34 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 00c523ca83a9caf824d6a1fafbc7ddff1f31ac5d, data reload: false

query1	0.02	0.02	0.03
query2	0.07	0.03	0.03
query3	0.25	0.05	0.05
query4	1.79	0.07	0.11
query5	0.55	0.51	0.52
query6	1.24	0.61	0.62
query7	0.02	0.01	0.01
query8	0.04	0.03	0.03
query9	0.51	0.49	0.47
query10	0.53	0.53	0.53
query11	0.12	0.09	0.09
query12	0.12	0.09	0.09
query13	0.62	0.62	0.61
query14	0.77	0.78	0.80
query15	0.77	0.76	0.76
query16	0.39	0.36	0.39
query17	1.03	0.98	0.99
query18	0.23	0.25	0.25
query19	1.95	1.87	1.88
query20	0.02	0.01	0.01
query21	15.46	0.53	0.54
query22	2.12	2.23	2.24
query23	17.23	0.92	1.09
query24	5.27	1.08	0.94
query25	0.35	0.13	0.05
query26	0.58	0.16	0.15
query27	0.03	0.04	0.04
query28	7.86	0.72	0.73
query29	12.76	2.33	2.28
query30	0.61	0.54	0.53
query31	2.81	0.38	0.38
query32	3.43	0.49	0.51
query33	3.08	3.10	3.08
query34	15.24	4.78	4.78
query35	4.85	4.83	4.86
query36	1.05	1.01	1.02
query37	0.06	0.05	0.04
query38	0.03	0.03	0.02
query39	0.02	0.02	0.01
query40	0.16	0.14	0.14
query41	0.06	0.02	0.01
query42	0.02	0.01	0.02
query43	0.03	0.02	0.02
Total cold run time: 104.15 s
Total hot run time: 31.34 s

@doris-robot
Copy link

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit 00c523ca83a9caf824d6a1fafbc7ddff1f31ac5d with default session variables
Stream load json:         20 seconds loaded 2358488459 Bytes, about 112 MB/s
Stream load orc:          58 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet:      31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select:       21.7 seconds inserted 10000000 Rows, about 460K ops/s

@starocean999 starocean999 merged commit 4818ba0 into apache:branch-2.0 Sep 14, 2024
21 of 23 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants